Embodiments disclosed herein relate to deep question answering systems. More specifically, embodiments disclosed herein relate to an inner passage relevancy layer for large intake cases in a deep question answering system.
In some use cases of a deep question answering (deep QA) systems, like IBM's Watson®, cases (or questions) and the passages being compared are often of dramatically different sizes. For example, in an insurance provider implementation, a corpus consisting of various medical insurance policies and guidelines may exist, which in turn consists of specific criteria corresponding to a given procedure. In order for a requested procedure to be approved by the insurance provider, the specific criteria must be met in accordance with the patient's clinical information (CI). The patient's CI often contains detailed history and description of present illness which can span several pages of text. The CI is essentially the question (also referred to as an intake case) sent to the deep QA system. To counteract the large intake case, the deep QA pipeline may turn the policy and guideline criteria into the “question,” and the intake case into the supporting/refuting “passage.” This allows the relatively short criteria to be the subject of deep analysis by the deep QA system. However, despite this switch, many scoring algorithms of the deep QA system still perform poorly (in terms of accuracy and computational performance). For example, a passage term match scorer attempts to match each term in the question by comparing it to each term in the passage. It then produces an overall score by totaling all the individual question term scores. If the intake case (i.e., the passage) is substantially larger than the criteria (i.e., the question), then the chances of every term in the criteria matching something in the intake case abnormally increases. The problem is exacerbated by a lack of focus in the question. In the insurance context, there are only three possible answers to an intake case—approve, deny, or refer to physician. It never makes sense to replace part of a criterion with any of these answers to form a hypothesis in the traditional sense. Embodiments disclosed herein propose a novel solution to this problem.